Papers with causal tracing
Correcting Language Model Outputs by Editing Salient Layers (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing models that use manual layer selection require prior domain knowledge and expensive empirical layer selection methods. |
| Approach: | They propose a model editing approach that selectively edits a small subset of model parameters to update the factual knowledge. |
| Outcome: | The proposed solution matches the accuracy of previous approaches with only 1/3 of their edits, enabling efficient updates to the parametric knowledge in large language models. |
Tug-of-war between idioms’ figurative and literal interpretations in LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | We find that idioms have non-compositional figurative interpretations that diverge from the idiomatic literal interpretation. |
| Approach: | They employ causal tracing to analyze how pretrained causal transformers deal with idiom ambiguity. |
| Outcome: | The proposed model leverages the idiom's context and refines it if it conflicts with the retrieved interpretation. |
CoSToM: Causal-oriented Steering for Intrinsic Theory-of-Mind Alignment in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models lack intrinsic cognition and cannot generalize to complex task-specific scenarios. |
| Approach: | They propose a framework that transitions from mechanistic interpretation to active intervention to map internal distributions of ToM features and implement it via targeted activation steering within ToM-critical layers. |
| Outcome: | The proposed framework significantly enhances human-like social reasoning capabilities and dialogue quality. |
Promote, Suppress, Iterate: How Language Models Answer One-to-Many Factual Queries (2025.emnlp-main)
Copied to clipboard
| Challenge: | To answer one-to-many factual queries, a language model must simultaneously recall knowledge and avoid repeating previous answers. |
| Approach: | They propose a promote-then-suppress mechanism that enables LMs to recall all answers and suppress previously generated ones. |
| Outcome: | The proposed model first recalls all answers, and then suppresses previously generated ones. |
Fact Recall, Heuristics or Pure Guesswork? Precise Interpretations of Language Models for Fact Completion (2025.findings-acl)
Copied to clipboard
| Challenge: | Language models (LMs) can make a correct prediction based on many possible signals in a prompt, but not all corresponding to recall of factual associations. |
| Approach: | They propose a model-specific recipe for constructing datasets with examples of four different prediction scenarios: generic language modeling, guesswork, heuristics recall and exact fact recall. |
| Outcome: | The proposed model-specific recipe yields distinct results for each scenario. |